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A project to help vocational-technical teachers in 
the development and experimental analysis of self- instructional 
programs is presented. The emphasis in developing the program was on 
maximizing effectiveness and efficiency of program-learner 
interaction as measured by criterion items. These items emphasized 
cognitive content dealing with the knowledge, application, 
comprehension, and analysis levels. The criterion items also included 
manipulative content, emphasizing the graphic solution of kinematics 
problems. Relevant technical literature was reviewed, and technical 
experts consulted. The analytical methods and guidelines used to 
determine program content, to discover strengths and weaknesses, and 
to determine the effectiveness and efficiency of the program have 
proven to be effective. Conclusions include: (1) Content for 
self-instructional programs can be determined by identifying a domain 
of criterion items and selecting those items appropriate for a 
specified target population; and (2) Th6 inclusion of randomly 
selected test items in the self -instructional program indicates to 
the student what is expected of him in the manner of his response to 
terminal criterion items. (CK) 
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THE DEVELOPMENT AND 
EXPERI>ENTAL AIvIALYSIS 
OP A SELP-INSlRaCTIONAL 
PRCDGRAM IN GRAPHICAL KINEMATICS 



Problem 



The general objective of this project was to help vocational-technical 
teachers in the development and e:q)erlinental analysis of self -instructional 
programs. The specific objective of the project was to develop a self-instruc- 
tional program and to verify its effectiveness and efficiency. In order to 
meet this specific objective, the project developer identified a number of 
enabling objectives. They were to: (1) detemdne appropriate subject matter 
content for a program in the area of graphical kinematics; (2) find effective 
methods for the development of a program by the manipulation of indeoendent 
and dependent variables; (3) develop analytical orocedures for the identifi- 
cation, and ijiprovement of inadequate and ineffective portions of a pnDgram; 
and develop empirical guidelines for writing and evaluating pr^'cpms of 
Instruction. ^ 



Significance of the Problem 

An ijiiportant objective of vocational-technical education has been the 
iTiprovement of instruction. The need for work in this area becomes very 
af^arent as we study professional journals and note the increase in sub1e-t 
matter content without an additional increase in time allotted. Howe l5l 
observed this developiient when he wrote: 

For most of us, the time allotted for teaching [engineering] 
graphics has been gradually but materially reduced, as the many 
other educational essentials have been worked into our curriculums. 
To meet the changing conditions, more efficient methods for teach- 
ing the most carefully selected materials, in strictly limited 
amount, is replacing our former well-founded but more time con- 
suming procedures. Thus we continually search for the effective 
means to maintain our standards and include the new ideas which 
are constantly appearing [5, p.22j. 

Many times students do not have the opportunity to see or hear well 
planned presentations by teachers. Class size limitations and the contact 
houre assigned to each instructor prevent contact of many students with the 
outstanding instructor in many cases. 

^ ^Self-instructional programs, however, can increase the teacher's personal 
contact with the students. Teachers typically cannot deal adequately with lar^e 
heterogeneous classes of students. Many students are often penalized by a lack 
of systematic reinforcement and active involvement with subject matter. Deter- 
line 12 J contended that; ^'^^t. 

The students at the extreme upper and lower ends of the con- 
tinuum of learning aptitude are usually additionally penalized 
because of the difference between the pace set by the teacher 
for the class and the .rate of subject matter consideration that 
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would be best for each individual student. The self--paced feature 
of auto-instruction (each student proceeding ac his own race) can 
at least partly eliminate the problem of the slower and faster 
learners who cannot adequately he taught by presentations designed 
for, and aimed more directly at the middle of the group [2, p. 74]. 

Various self -instructional programs could be made available to students 
whD may have difficulty understanding or who were absent for the initial pre- 
sentation. Other instiructors could avail themselves of various prepared self- 
instructional programs for use in their classes. SubctiuUte instructors could 
naintain continuity in their progress toward the course objectives. 

A need nov; exists for studies to be conducted dealir;g with an experimen- 
tal development and analysis of self-instructional programs which Sxhould require 
enpirical evidence as to program effectiveness. 

Lumsdaine [6] is convinced that: 

...most existing programs afford only a rough approximation of 
the potentiality for control over learning which could, in prin- 
ciple, realize a goal of assured mastery for all qualified students 

Even casual inspection of a sample of programs suggests a 
tendency merely to follow superficially the general format implied 
by one programming rationale or another, while meeting neither the 
theoretical assumptions or empirical characteristics that are sup- 
posed to be exemplified. In addition to lack of adequate tryout 
and revision, many other apparent weaknesses are to be seen in 
examining the existing programs, including inadequate analysis of 
subject matter content and inept use of what seem to be the more 
promising techaiques of programming [6, pp. 271-2]. 

Self-instructional programs are to be ideally develcoei within the con- 
text of a laboratory experiment. Hively [3] listed basic methodological prob- 
lems Involved in laboratory experimentation. They are: 

(1) the problems of recording dependent variables (2) the 
problem of replicating independent variables from one experiment 
to the next, and (3) the problem on controlling other con- 
ditions which might affect the system which is being studied. 

In general, the experimenter's problem is to seek out and demon- 
strate orderly and reliable effects of his sec of independent 
variables upon his set of dependent variables. To the extent 
that he finds them he has laid the groundwork for an experimental 
science of instruction [3, pp. 1-5]. 

All of these considerations sei-ved as the basic rationale for the carrvine 
out of the project. ^ 



Procedui^ 



Tne project was developmental in nature in that the metnodo and procedures 
for developing a self-instructional program m graorilcal kmeniatics were studied 
^ and reported on from the initial criterion item soecification to the final deter- 

ERXC "•ination of the program's effectiveness and efficiency. Fx^ror's 1 xLlu^trates the 



systems approach used in developing the seif-instructicr.al prc^i-^un. 

In developing the self-inst^ructioml procrain, tre enxhalis v.as on max- 
imizing effectiveness and efficiency of prcEran:;-learn5r interaction az measured 
by criterion xcems. The criterion items enphasized -cogTiizive corxenr dealing 
with the knov/ledge, application, conpreher^slon, and araiysis leveis. The 
criterion items also included iranipulative content, erf:ptu"izin.j, cne ^-raphic 
solution of kinematics pi-oblems. " 

During the project an attempt was made to define an exhaustive list of 
criterion items for the unit tau^t. From this list of cr-i:eiion items, a 
sanple vas selected for use in constructirig '.he 3eif-initi-..ction p-oaram and 
the criterion test. 

The terminal criterion behavior was a statement of a teaching objective 
in terms v/hich ailoved the existence of the behavior to ce tested arpirically. 
Bipirical testing yielded information wnich alxcwed the ^:eli-in=cructional 
program to be improved and revised. The sp jcif ication and araiysis of terminal 
behaviors was a major activity in che preparation of che self-instruction pro- 
gram. 

In order to acconplish the first step, the relevant technical literature 
was reviewed and a number of educational and technical experts v;ere consulted 
in order for the project developer to master the subject natter. This aspect 
of the project has been illustrated as Event III in ri^iiu-e i. 

The termirial criterion behavior desired upon oa^ir.factox-ily completing 
all the frames in the self -instructional program booklet was established as the 
100 percent ability of the reader to: (1) Recognize a feauceilier's m.echanism; 
C2) recognize a rtodified Peaucellier's mechanism; (3) recogrJ.ze an inverted 
Peaucellier's mechanism; (^) indicate selected link ratios required in order to 
have a Peaucellier's mechanism; (5) state v/hich lirJcs in a Peaucellier's mechan- 
ism, a modified Peaucellier's mechanism, and an inverted f eaucellier 's mechanism 
have linear or angular motion; (6) state which linkage ratios cause point path 
curvature in a modified Peaucellier's mechanism; (7) indicate selected point 
relationships as motion is introduced into a Peaucellier's mechanism, a modified 
Peaucellier's mechanism, and an inverted Peaucellier's mechanism; and (3) graph- 
ically analyze a Peaucellier's mechanism, or a modified Peacueliier's mechanism 
or an inverted Peaucellier's mechanism. 



The Target Population 

At the time the project developer was establishir^g the ret^uired terminal 
behavior, consideration had to be given to identifying zhe target population. 
This is illustrated as Event II in Pig-ure 1. It was emphajj.sed in the litera- 
ture that the development of a valid training and learning system requires the 
specification and contx-ol of target populations. Ths specif icatiors should 
cover more than just the general psychological and IntelleLtuai characteristics 
of the student. The prerequisites should be listed and ;~u6llec cit in benavioral 
terns as shov/n in Figure 2, Item^s (P1)-(P7). Better toeoliication ci the target 
population, with pretesting to insure the existence of prersoui:.iCe behavior 
should yield .mch mere efficient educatior^l programs. The o"c;: .ilaticn for the 
project con.l:,ted of student i^row^i enrolled m varic^j oo:,--.;-. ".oraa.-v design 
and drafting co^^es. 
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Sequence of the Behavior Network 



Event IV as shown in Figure 1 a?epresents the arrajigemenf of behavioral 
objectives from which the self-lnstructiorjal prcgram was developed. Pigiu'e 2 
illustrates a schematic diagram showing the realtionshlps of the elements in 
the hierarchy of graphical IcLneratlb objectives for Peaucellier's mechanism 
Oisplacement. The formal structure of the behavioral nefr/fork served both as 
a foundation and a practical constraint to the self-instructional program ' 
developer. The development of the behavioral network required the establish- 
ment of terminal criterion behavior, mastering of the subject matter, and 
determination of the target population. ■ 

The prerequisite criterion item coding key follows (selected examples 
only): *^ 

(PI) This item consisted of the demogr^hic deocription of the target 
population. The population had conpleted a basic course in 
drafting or engineering drawing. This single item assumed items 



(P3) Make lineai" measurements within l/50th part of an inch utilizing 
a scale with inches divided into decimal parts; 



(P6) Use a protractor to measure angular displacements to within an 

accuracy of fifteen minutes; and 
(P7) Develop basic geometric constructions in the transfer of angles 

and lines. 

The Intemediate behavioral criterion item coding key appears as follows' 
(8) Identifyin/. kinematic foims; 



. (32; Indicate which linkage ratios cause straight-line point path 
movement in an inverted Peaucellier's mechanism. 
The terminal behavioral criterion item ceding key is as follows: 
(T33) Indicate selected linkage point relationships as motion is intro- 
duced into the mechanism; 



(T42) Graphically analyze an inverted Peaucellier's mechanism. 

The beha^/loral planning network took into acco'ont not only behavior 
itself, but also the conditions under which the behavior occurred. As a result 
the conditions under which the student received all program materials were judged 
to be similar. The environmental factors were those for tj^pical college drafting 
classroom conditions. These conditions tend to simulate actuil workins; environ- 
ments in design and draiting occupations. 

nrv^nooHf^'f :\f Je-^-vic-^^l P^Lsnning network was established, the project developer 
proceeded in the development and writing of the criterion test instruments and the 
program frames. 
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Development of the Criterion Test Instruments 

The development of a conprehensive criterion test was essential and has 

T^rifSnt A test always IncorpoStefa 

sufficient statement of terminal items, intemediate, and often prereauisite 

Si cS^S n? l^t'^^^'^^^^^f^ °f the final criterion test. All the technology 
ana caution of test construction was required. 

There were no applicable standardized tests for measuring the ability 

S STf.'^'i'%''f P?^'^' ^"•'^'^^ tasks, ^^e studentl we?e requiLd 
oo Identify selected mechanism characteristics and to produce graphical solutions 
in meeting the requirements of the criterion behavior. S^^pnicai solutions 

/ov , project developer considered factors such as: CD content vaiiriltv 

Development of Frames 

4 ^ The program variation that was used in developing the project was deter- 
mined by the criterion performance reouir^ments . Project was deter- 

Knowledge conprehension, application and the analysis of graphical 
kinematics concepts require verbal behavioral outcomes for practiS use in 

X^^J^^r'''^^''' "^"S ^^P^^^l construction^ cSpStion 

and constructed responses approximated most closely the desired behavioral 

S^f^nt^p^ol^ss'^^ ^' "^"^-^^^ S?3^ 

because^S Se^H^f^f variation selected was the linear type 

because of the desire for error , response and the ease of diagnosing pos- 
sible sources of misunderstandir^ .y the student. It could be besTdSKmd 
to cover the necessary material in the IdMted time available. ^^^^Sned 
• ^v,^ The self-instructional program format selected was booklet form because 

teristics. The booklet had only a single stimulus unit, followed by a comole- 
quentillJr ' "'"^ °" '"'^ werf nSoerS^e! 

The efficient and successful development and debusslnff of the self- 
instructional program materials required field testlnS i^^te fon^ of S 

neTS'dlmctn^ oT^h'^ ^1'^' for^r^lSli^Thf e??ectlve- 
"X'• s W?S self-instructional program. In the illustration an 
cnH nn^ ^^'^ exposed to the self-instructional program and 

Field Testing Experimental Design 

nii^f r^^^^S^ development and experimentatlcn was accoin^lished by use of a 
ri^eST^eetelX^i Slij^' ? '■'^ '^^^ ^^^^ 

Minnesota, Minneapolis, I41nnesoca. Group III was oomrlSd of ,fnI»J^ffL i, ^ 
in .echanlcal drafting at M.„e sJe^'/^r St^Sl^'S! SSesola. 
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Group IV consisted of students enrolled in nechanical drafting at Willnar Vocational- 
Technical School, Willmar, ^aJlnesota. Groups V and VI consisted of students enrolled 
in two descriptive geon»try classes at Stout State University, Menononie, Wisconsin. 

The students in each group received pretests, the self-iristructional program, 
and a posttest as an integrated part of the respective course requirements. The 
reader should keep in mind that Group VI did not receive the self-instructional 
program pretest, but received only the Space Relations Test and the techanical 
Conprehension Test. The deletion of the self-ir.structional Drogram pretest was due 
to the inability to schedule the same ttae period and length of tUne for Grouo VI 
t hat vas available to Groups II through V. The experi-nental treatment of the" self- 
Instructional program and the posttest were administei^d during the last two weeks 
or each course, while the pretests were administered approxL-nately tvro days before- 
the students received the program treatment. The pretest and cost test consisted of 
identical, but rearranged criterion items. 

In an attenpt at conplete optimization, the re3cor^e errors on criterion 
items and the time students spend in self-instr-uction are rlnimised. The decree of 
retention after a specified period of clme, as measured by errors on criterion itons 
snould also be considered. 

Klvely suggested the use of a person-by-item matrix, as shown in Figure 
for use In estimating the easiness of a single criterion item. 

PERSONS ITEM 1 



FIGURE H 
PERSON-BY-ITEM MATRIX 
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In this matrix, person p receives a score which equals 1 if he answers 
correctly and 0 if he answers incorrectly" [/{, p. 8]. The easiness (M^) of the 
specific Item, or unit , can be calculated by M^ = IXp . 

as In F1^^5^^^"^ perfonrance over a set of items, the data may be sunmarized 

4.^^ ^M^^^i^" °^ ^^^^ answered correctly (M^) by person p, can be de- 
texTtlned by Mp = iXpi , and the mean for the entire item form(r4f) can be calcu- 

i 

lated by Mf = ZZpi . in the development and revi:;ion of a seir-lrutruotional urc- 

&^m, the conparison of M^'s, M^'s and M^^'j for diiferent verciorr. should uiwlde 
w - clues to the effectiveness of the vai'iouo program ibnn::. 
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FIGURE 5 
PE31S0N-BY-ITEM I^TRIX 
The personr-by-item matrix may also include a doubt distribution. 



Units 
Students 


"l 


U2 


"3 


Ui, 




"6 


« • • 


1 

Un 


Si 




E? 




9 




« • • 




S2 


E? 




E 




0 




• • » 










E? 








• « « 




Sii 






E 




0 




« • • 




S5 






E? 








• • • 


















• • • 




• 
• 
• 


• 
• 


• 

• 
• 


• 
• 


• 
• 
• 


• 


• 
• 


• 


• 
• 
• 


Sn 






E 




9 




• • • 





Reading Key; E em^r ? lack of understanding 

In this simple hypothetical example we' find a clear error concentration 
on Unit 3 on the one hand (for almost all subjects) and for Subject 2 oi 
the other hand (for almost all units). Va also find a rela;;ively large 
number of question marks for unit 5 in spite of the fact that this unit 
has not attracted any errors at all. These kinds of Infonration are 
evidently better starting-points for revision attempts than overall ern 
percentages for total programs. .. [l, p. 120]. 

FIGURE 6 

TYPICAL STRUCTURE OF A SIMPLZ SURVEY TABI£ 
('»ERROR-ANI>-!:OUBT DISTRIEOTION" ) 
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The student's terminal behavior should be the final criterion of the effec- 
tiveness of a self-instructional unit. If students c^n reach acceotable levels 
of understanding within a reasonable time span, the progra-n can be* termed efficient. 

The Corrputational Procedures and Defining the Pooulatlcn 

To aid in making a valid analysis of the students' resccnses to the orogram 
and posttest, basic information was needed pertaining to the equivalency of the 
selected groups. If Groups II, III, r/, V and \1 were found to* reneraily beloao: to 
the same population, a more valid ccnparison of progr2.11 and oosttest iM, and Mf coef- 
ficients could be made for each revision of che self-msrr-uclional proiram materials. 
Scores earned on: (1) the Graphical Kinematics Pretesc; (2) the Eenr^tt ffechanical 
Comprehension Test; and (3) the Space Relation Test (D/il) i-rere used tc ccrroare tr- 
groups. Graphical kinematics pretest conpletion times and zhs nuirJber of vieeks of 
drafting training were also used to conpare the grcuDs In order to detemine v-hethe" 
the participants from the four locations vrere of the same poouiation. In onier to 
compare the groips certain afpropriate statistical tects v/ere utilized. 

• One-way classification analysis of variance techniaues as suggested by 
Popham [7] were used to determine the significance of mean differences between the 
groups sim.ultaneously, for each of the five ineasures previously specified. Tvrro 
basic assunptions of the one-*'ay analysis of variance' we:'e that t'ne subercups 
randomly dravm from a normally distributed target peculation and that the vkriance 
within these subgroL?)s be homogeneous. 

Popham C7] indicated that when taking a theoretical viev.'ooint the assu-rotions 
underlying analysis of variance must be rigorously fulfilled. This researcher goes 
on to say that there is evidence that even if fairly significant departures from 
strict theoretical assunptior^ may exist, analysis of variance is sufficiently 
"robust" so that ir will give results v/hich can be meanir^fullv interpreted. 

Where analysis of variance produced a significant F ratio indicating th^at a 
significant mean difference existed between groups, the project develocer utilized 
the analysis method proposed by Scheff4. This method, as presented by Walker and 
Lev [8, pp. 303-6], leads to a confidence inter\'al for a contrast antDng population 
means. The confidence limits for the contrast Ui - ui are: f ^ 



Upper limit: (Xj^ - Xj) + (k-DF^il^^xl^ t i ) 
Lower limit; (Xi - Xj) - (k-l)P^(r%)(l i- i 



'J 

when: (Xj^ - Xj) = mean difference between contrasted groups; 

k = number of grotps; 

Pc = c th percentile for (k-1) and (r.-l) degrees cf 

freedcrr. obtained ircm t:;e i.zlor oniilycis cf variance; 

MSy = error mean square obtained ii-an the prior analysis of 
Q variance; and 
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n^ and nj = number m each respective ci.r.trajted j:roup, 
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In ^plying these formulas, if one limit was fcsnd ro be positive and the 
other regative, 0 was a possible value of - y j , so -he obier-.e-d difference 
(Xj^ - W was nonsignificant. As a result, it was concluded that the population 
means the two contrasted groups did not differ siznii leant ly from each other. 

The project developer t.'ien proceeded to test foi- hoiicgeneity of variance 
among the subgroups. Pophain [7] and Walker and Lev [8] indicate that Bartlett's 
test can be used to derennine homDgeneity of variarxce a-rorig 3ubgix)UD3 of different 
^anple sizes. Bartlett's statistic (B) «ith a critical i^gion B >x^a:k-l is: 



B = 2.3026 



Z (ni - 1) log - Z (n, - i) log sfl 

i=l J 



i=l 



when: is the pooled variance of the k sanple variances 
and 



i=l and when: 

"k 

r (r.. -1) 
i=l 

r = 1 + 1 ^ 



i=l 



ni-.l 



r (n. - 1) 
1=1 



If B was less than the critical region set up for x there was statistical 
evidence that the population variances were eaual. By u=;e of this statistical test 
the project developer was able to test for a basic assaTi^tion underlyirig tlie 
analysis of variance technique. * 



Graphical Kinenatics Pretest 

The data used in conputing the various statistical tests are shovm in Table 1. 



TABLE 1 

DATA BASED ON GRAPHICAL KB-iBlATICS PREISS'I 



Groip Location n J " ^ 



Group II 22 2.27 l7l6" 

Group III il\ i.ij "l^g 

Group r/ 10 1.90 77 

Group V 18 1.33 6.5S 
Groi^ VI (Did not participate in graphical kinerrstic:; px^erect; 
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A sunmary of the statistics used In the analysis of variance for the 
graphical kinematics pretest spears in Table 2. 

TABLE 2 



ANALYSIS OF VARIANCE FOR GRAPHICAL KH-EMATICS PRETEST 



Source of 


Sum of 


Degrees of 


Mean 






Variation 


Squares 


Freedom 


Square 


F 


P. 95 


Between groins 


12.52 


3 


4.17 


l.?i|- 


2.76 


Within grotps 


187.48 


60 


3.12 






TOTAL 


200. 00 


63 









The critical region at the .05 level of significance and with the appro- 
priate degrees of freedom was F>2.76. Tne calculated F value indicated that there 
was no significant difference among the means of the four groups. 

In testing for homogeneity of variances for the grouos, the project develop- 
er utilized Bartlett's statistic. The rejection interval for the graphical kine- 
natics pretest was X^>xl(^^g^y The tabulated critical region was x2>7.8l5 and the 

calculated value of B was 18.97 indicating that the gr-oi?) variances were not honc- 
geneous. 



Space Relations Test (DAT) 

The data used in calculating the various statistical tests based on the Soace 
Relations Test (DAT) are shown in Table 3. 



TABLE 3 

DATA BASED ON SPACE RELATIONS TEST (DAT) 



Group Location 


N 


1 


s2 


Groip II 


22 


77.68 


139.80 


Grovp III 


in 


74.43 


381.80 


Group IV 


10 


81.20 


122.40 


Group V 


18 


77.17 


238.45 


Group VI 


21 


77.63 


219.29 



o , .4 ^ ^i!™?^r^?£^^^® Statistics used in the analysis of variance for the Space 
ERIC Relations Test (DAT) appears in Table 4. ^ 



in 

TABLE H 



ANALYSIS OF VARIAIvICE FOR SPA'CE 
RELATIONS TEST (DAT) 



Source of 


Sum of 


Degrees of 


Mean 






Vaidation 


Squares 


Freedom 


Square 


F 


P.95 


Between groups 


520.86 


n 


130.21 


9.28 


2.il8 


Within groups 


1,122.25 


80 


1^.03 






TOTAL 


^1, 6ii3.il 


an 









The critical region at the .05 level of significance and v/ith the appro- 
priate degrees of freedom v/as F>2.i|8. The calculated F value indicated that there 
was a significant difference among the means of the five groups. 

In order to determine which sanple mear^ of the groups" tested v/ere signif- 
icantly different from each other, the project developer arranged the groups in 
order of increasing magnitude. All possible differences between cairs were then 
determined and presented in Table 5. 

confidence intervals v/ere constructed for all possible unweighted contrasts 
between pairs by use of Scheffe's method. The confidence llrplts for Groups ril 
and IV were found to be +1.89 <Mn - Mo< +11.55. All other conficence limits in- 
cluded 0 as a jwssible value of - p.. The project developer concluded that the 
mean scores for Groins IV and III differed sigrlficantly from each other. 



TABLE 5 



DIFFERENCES BETVEEN PAIRS FOR 
SPACE RELATIONS TEST (DAT) 



Group 


GROUP MEANS (X) 




III 


.V ■ VI 


; "11 


IV 




X = 




77.17 77.63 


77.68 


81.20 


III 






2.7^1 3.20 


3.25 


6.77* 


V 






M 


.51 


k.03 


VI 








.05 


3.57 


II 










3.52 


IV 













*Significant 



In testing for homogeneity of variances for the groups, the project developer 
obtained a B value of 5.^*6.. The tabulated critical region was found to bc^ x^>9.^88 
which provided statistical evidence that the group variances were homogeoeour at tne 
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.05 level of significance. 



Graphical Kineniatlcs Posttest 

ccinpletion of the graphical kinematics self-instruction pro-am t'-^e 

SSi^cH^n^'l.^'"'' '° ^ "^^^^"^ -^i^^ S^^Phical coSSgfand 

constructions. The average scores and group variances are shown in Tkble 6. 



TABLE 6 



DATA BASED ON GRAPHICAL KOJET'iATICS rCSTTEST 


Group Location 


N 


X 


s2 


Groip. II 
Group III 
Group IV 
Group V 
Group VI 


22 

13* 

10 

18 

21 


9.00 
10.31 
10.90 
11.61 
11.91 


6.38 
h.S9 
5.66 
6.13 
1.19 


*One student did 


not participate in postcest (pretest U=lH) 





The project developer was encouraged by the steady increase in the svpr^o^ 

l^ Jl fo^ tfT '"^^ '^^"-^ ^^^^^ '^^^^^^y inci^ased. TMs w4s 

WeS foS ^^'^P' J"^^ ^' P^^^^^ ^^^i^^^ f-^^ Group V was t^f 

SSSe^^ wS-^"*' receiving the pretest.- The posttest variances tended to 
increase from wha. the pretest variances were for each group receivins; both tests 

if fnSi^JS posttest variance. The groi?)s' posttest variances slnply tended 
to indicate the degree of individual differences within each group. 
Dr^test^rn^^fi-i? PJ'^^^^f d in Table 1 and Table 6 were coiri)i;£d. the average 
tSes these g2S variances were found. Table 7 illSs- 
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TABLE 7 

DATA BASED ON GRAPHICAL KINETTOGS PRETEST TO POSTTEST GABi 




Group II 
Group III 
Group IV 
Group V 



22 

13* 

jlO 

18 



6.73 
9.08 
9.00 
10.11 



Group VI (Did nor participate in graphica l kinemati cs ' pretest ) 
«One student did not participate in posttest (pretear N=li;) 



7.26 
13.63 
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The data presented illustrates a steady average gain in scores for each 
revision of the self -instructional program. Even though the pretest to post- 
test average gain for Group V ^vas the greatest, the pretext to posttest gain 
variance v/as approximately three times the gain variance for the previous 
revision. The pretest to posttest gain variances tended to indicate the ainount 
of indivld'Jial differences for each group. 

The average conpletion times in minutes for the posttest are showTi in 
Table 8. The average times remained ^^latively constant, while the time variances 
for the posttest indicated deer-easing values. 

TABLE 8 

DATA BASED ON GRAPHICAL KINBIATIC POSTTEST TIMES ' 



Group Location 


N 


X 


32 


Group II 


22 


15.68 


31.46 


Group III 


13* 


1^.53 


30.10 


Group IV 


10 


14.10 


10.29 


Group V 


18 


15.06 


20. C: 


Group VI 


21 


14.29 


16.41 
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*One student did not participate in posttest (pretest N=li|) 

Conparison of Item Mj 's, M^'s and Doubt Distributions 

During the development and experimental a-ialysis of the self-instinictional 
program, the project developer was required to ccllect certain data. Tnis data 
provided the basis for any revision of the prograii format, sequence and content. 

Tables 9y 10 and 11 sumrarize corrpletely the degrees of easiness for each 
criterion item (Mj_*s) from the pretest and posttest. The average times in min- 
utes for the constructed i^sponse. Item 1 and the total average times in minut^s 
for criterion Items 2 throu^ 13 are also given for' each program revision. The 
degrees of easiness for each test (Mf 's) indicate any chai';ge in performance for 
each participant group. The degree of doubt for each item and for each group 
is presented as a doubt distribution (?). 

In analyzing the posttest values for Revision I, Table 9, which was 
administered to Group II, the project developer deteiinined that the sequence 
leading to criterion Items 1, 2, 3, 4, 5, 6, 7, 8, IC and 13 required revision 
in oi\3er to increase their effectiveness. The posttest Mj_ values for Revision 
II, Table 10, Groups III and IV indicated that the sequences leading to Items 
1, 2, 3, 4, 6, 8 and 13 required still further revision. Itens 3, ^ and 8 of 
Revision III, Table 11, Groups V and VI still require additional program sequence 
revision. 

The doubt distribution for the pretest and posttest included only doubt 
responses (?) for those items which were answered correv;tly vvith doubt* Tho.^e 
items that v/ere answered incorrectly with doubt ;vere ret incluciea in the dis- 
tribution. The degree of doubt tended to decrease for each program revision. 
The doubt decreased from pretest to posttest except for Group TJ. The relatively 
high doubt distributions tended to correspond to the low values. 
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The criterion test Mf values for the three revisions steadily increased 
from approximately .60 to .90. These values indicate a trend towards 100 per 
cent program effectiveness. 

The percentage gain for each program revision was calculated by dividing 
the total pretest to posttest gain by 13 for each Lr:diviJual and then detennining 
the average percentage gain for each group. The percentage gain steadily in- 
• creased for each program revision. 



Program Informtion Density and Informational Speed 

During e:^erimentation utilizing the program mateidals, the project develop- 
er calculated infornEtional deriSity and informational soeed coefficients. The 
values obtained are given in Table 12. 

The information density coefficients (I^) were obtained by using the follow- 
ing formula: 

Id = Vi% 

when: 

= the average error perj^ntage of the student group on criterion problems 
before using the self-instructional materials; 

Bj^ = the average error percentage of the student group on criterion problems 
after using the self-instructional materials; and 

Dj^ = the number of program responses used in teach.lng the selected concepts. 

The information speed coefficients (Ig) were conputed using tMs fomula: 

Is = ^1-^1 

when: 

Ti = the average program time in minutes. 



Fulfillment of Objectives 

The self-instructional program on graphical kinematics experimentally 
developed and tested during this project did not reach the ultimate goal of 100 
percent correct perfonnance and, therefore, needs continued inprovement as it is 
used for future Instmction. The results indicated, hov/ever, that by utilizing 
the methods of content development and analysis used during the project, an 
effective and efficient self-ir^:tructional progr-am car. be developed. The major 
objective of the deveicpmental project was basically luiiiiled. 

The matrix analysis techniques used for determining the s:-i>ep-gths and 
weaknesses in the self-instructional program and the use of iniormational den- 
sity and speed coefficients proved to be effective methods in aralyzing students* 
responses to the progtcuii booklet and the criterion tests. The informational feed- 
back from the quest iorjialre scheiule and from individuals participating in the 
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pilot study also proved to be of value in determining troublesonie segnents 
of the prDgram materials. 

The analytical methods and enpirical guidelines used to detenrine program 
content, to discover strengths and v;eaknesses, and to determine the effectiveness 
and efficiency of zhe program have proven to be effective. These, same methods 
should be able to be effectively used by teachers and educational technologists 
for developing self-instructional programs in other subject areas. 



Conclusions 

During the period that the project developer was involved with this project, 
certain data were gathered and organized. From this data the following conclusions 
were drawn: 

Content for self -instructional programs can be deteraiined by identifying a 
domain of criterior^ items and selecting those items appropriate for a specified 
target population. These selected criterion items must define in exact behavioral 
tenns what the student should be able to do when he has been exposed to the pro- 
gram of instruction. 

Programs of instruction can be constantly inproved by analyzing the responses 
made to the program booklet and the criterion tests. Tliis can te acconplished 
only if some method of coding is used to identifV selected progt^am micro sequences 
with the ^propriate criterion items. 

Items selected from a domain of criterion items v/ere used to test the 
student understanding of the subject area. This process can be used to determine 
both individual and groip performance and also to locate inadequate program and 
criterion test segnents. 

The inclusion of randomly selected test items in the slef -instructional 
program indicates to the student what is expected of him in the manner of his 
response to tenninal criterion iteins. Pretesting and posttesting can be accom- 
plished by randonly selecting criterion items from the dcxnain of criterion items. 
By doing this, the developer can determine the educational gain for individuals 
and groi^^s. 

During all phases of program development much relevant information can be 
obtained by recording students* conrBnts. Many program inprovements can be made 
by utilizing this information. 

The degree of effectiveness and efficiency for a program of instruction 
can be determined by the matrix analysis and analytical models used during the 
development of this project. 
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